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(57) Abstract: Means and a method for authenticating a photographic image (3) on an identification device (1), the identification 
device (1) being provided with: a photographic image of a person (3) and a microprocessor (8), the microprocessor (8) having: a) 
a processor (7), b) a memory (9) connected to the processor (7) and having stored authentication data, and c) interface means (5) 
connected to the processor (7) for communicating with an external device, wherein said photographic image (3) comprises stegano- 
graphically hidden information, the content of which together with said authentication data allows authentication of said photographic 
image (3), the method having the following steps: a') scanning the photographic image (3) and generating image data, b') analyzing 
these image data in accordance with a predetermined image analysis procedure to derive said hidden information, and c') carrying 
out the authentication of the photographic image (3) based on the hidden information and the authentication data. 
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Secure photo carrying identification device, as well as means and method for 
authenticating such an identification device 

5 Field of the invention 

The present invention relates to a photo carrying identification device, like pass- 
ports, and (credit) cards used to identify persons, and thereafter authorize them to do a 
predetermined action, like entering a building, passing a boarder, carrying out an auto- 
1 0 matic debit transaction from an account, etc. 

Background of the invention 

The invention relates to the use of identification (ID) documents equipped with a 
15 picture of a documents holder, e.g., a driver's licence, or a plastic card having the size 
of a credit card, or a passport. In a common use of such an ID document, a human op- 
erator compares the picture on the document with the face of the document holder to 
assess entitlements sought by the document's holder based on credentials as defined by 
additional data in the document. A passport, for instance, gives access to a country 
20 based on nationality of the document's holder. 

A problem encountered with such documents is that they are frequently copied 
with false credentials or a false picture. 

A common solution to this problem is the application of physical tamper detec- 
tion methods such a sealing foil covering both the picture and the document, often 
25 combined with special inspection tools, like polarized light, to probe the tamper detec- 
tion method. However, the use of such inspection tools often requires a skilled opera- 
tor. 

Another possible solution, referred to in paragraph [0002] of EP-B 1-0,539,439, to 
tampering with the picture attached to the ID document is in using smart cards provided 
30 with a microprocessor having a processor and a memory. The memory in the card chip 
stores a digital copy of the picture on the card. A terminal is provided to read the 
content of the memory of the chip card and to display the stored image on a monitor to 
an operator. Then, the operator compares the displayed image on the monitor with the 
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face of the actual card holder. This solution may even obviate the need to attach the 
picture on the card itself. However, this solution requires costly display equipment 
which, amongst other reasons, has made this solution unsuitable in particular areas of 
industry which offers great potential to the use of smart cards, such as public transit 
5 systems where ID smart cards are sought as efficient improvement of traditional dis- 
count passes. 

A further problem encountered in ID systems is in protecting the privacy of the 
individual using the ED document Especially in case such an ID document is realized 
as an electronically readable smart card protection may be required from uncontrolled 
10 and/or unapproved collection of data identifying the individual and his or her use of the 
smart card. 

To protect the privacy of the card holder, cryptographic techniques, e.g., blind 
signatures, may be applied to the process of reading ID and credential data from the 
smart card. However, the use of pictures stored in a card memory and read by a ter- 

1 5 minal for display on a monitor to an operator in principle defeats such cryptographic 
privacy protection. In such a case, the terminal is not only able to collect uniquely and 
strongly identifying data about individuals, i.e. their pictures, but also the nature of this 
data poses an additional threat in which, for instance, the individual may be compro- 
mised through digital image manipulation techniques. 

20 US-A-5,748,763, column 58, line 24, to column 62, line 45, describes a method 

and an arrangement for enhancing the security of credit and debit cards. The arrange- 
ment disclosed has a computer arranged for receiving a digital image of the card holder. 
After having analyzed the digital image the computer generates a snowy image which 
is generally orthogonal to the digital image and adds this to the digital image to render 

25 an amended, unique image. The intended effect is to "texturize 11 the original digital 
image. It is not necessary that the snowy image itself is invisible to a person looking at 
the image. However, the image of the card holder may not be obscured by the snowy 
image. The amended, unique image is printed on the card. Moreover, the unique infor- 
mation is also stored in a central accounting network. 

30 In a steganographic embodiment the snowy image is such that it is hidden in the 

photographic image of the person on the card. More detailed information as to stega- 
nography can be found in US Patent 5,613,004 and the references cited in this docu- 
ment. For the sake of the present invention steganography will be understood to relate 
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to any method of obscuring information that is otherwise in plain sight. The informa- 
tion is hidden in another medium. It is used as an alternative to encryption. E.g., 
spreadsheets or graphics files could contain a text message invisible to an unaware 
person. People unaware of the hidden information will not recognize the presence of 
5 steganographically hidden information even if the information is in plain view. 

In US- A-5 ,748,763, referred to above, a scanner is provided to scan the card 
when the card holder wishes to use his card for a predetermined transaction, e.g., 
automatic payment from his account to pay for a product The scanner is connected to 
the central accounting network. By means of a secure communication protocol the 

10 image of the card scanned by the scanner is transmitted to the central network. The 
central network is arranged to receive the transmitted information and to authenticate 
the validity of the image on the card. 

Additional security to the known system may be provided by requesting the card 
holder to input a PIN during the scanning process. Moreover, additional security is pro- 

15 vided by letting a third party, during the scanning process, check whether or not the 
person trying to carry out a transaction with the card is the person who's photo is on the 
card. 

A disadvantage of the system and method disclosed by US-A-5,748,763 is that it 
is only to operate when a central network is provided having stored all unique images 
20 of all participating cards. 

Summary of the invention 



A first object of the invention is to provide a photo carrying identification device 
25 that obviates the need for such a central network. 

Therefore the invention provides an identification device provided with: 

• a photographic image of a person and 

• a microprocessor, 

the microprocessor comprising: 
30 > a processor, 

> a memory connected to the processor and comprising authentication data, and 

> interface means connected to the processor for communicating with an external 
device; 
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wherein the photographic image comprises steganographically hidden information, the 
content of which together with the authentication data allows authentication of the 
photographic image. 

Thus, the invention provides an identification device which are provided with a 
5 microprocessor, comprising the authentication data necessary to authenticate the pho- 
tographic image on the identification device. In other words, the key to authenticate the 
photographic image is in the identification device itself instead of in a central network. 

Such an identification device may be, for instance, a passport or a plastic identifi- 
cation card, like a smart card. 
10 In one embodiment of the invention the processor is arranged to cany out at least 

part of the authentication. To that effect the processor will cany out a program prefer- 
ably stored in the memory of the microprocessor. 

The authentication data stored in the memory of the microprocessor may be a part 
of the photographic image on the identification device. However, it may also be data 
15 related to the photographic image. For instance, it may be related to grey level, inten- 
sity distribution, or image entropy of the photographic image. 

A second object of the invention is to provide a terminal, which is arranged to 
communicate with the identification device of the invention to allow carrying out the 
authentication process required. 
20 In a first embodiment the invention therefore provides a terminal arranged to 

communicate with an identification device, the identification device being provided 
with: 

• a photographic image of a person and 

• a microprocessor, 

25 the microprocessor comprising: 

> a processor, 

> a memory connected to the processor and comprising authentication data, and 

> interface means connected to the processor for communicating with a terminal, 
wherein the photographic image comprises steganographically hidden information, the 

30 content of which together with the authentication data allows authentication of the 
photographic image, 
the terminal being provided with: 

• a picture scanner to scan the photographic image and to generate image data, 
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• a terminal interface allowing communication with the processor of the identifica- 
tion device, and 

> an image processor arranged 

> to receive the image data, 

5 > to analyze these image data in accordance with a predetermined image ah 

procedure to derive the hidden information, 

> to receive the authentication data from the memory, and 

> to carry out at least part of the authentication of the photographic image based 
on the authentication data and the hidden information. 

10 In this first embodiment, the authentication of the photographic image is either 

partly or entirely canied out by the image processor in the terminal. 

The steps necessary to cany out said authentication will, in practice, be stored in 
a terminal memory. In an embodiment of the invention, the way in which these steps 
are carried out depends on the authentication data received from the memory of the 

15 identification device. In such an embodiment, the authentication carried out by the ter- 
minal will depend on data received from the identification device itself. This makes it 
impossible to predict the actual authentication steps carried out by the terminal, which 
enhances the security. 

However, the security can also be enhanced in an alternative embodiment in 

20 which the processor of the identification device itself carries out at least part of the 
authentication of the photographic image. Therefore, the invention also provides a 
second embodiment of the terminal. This second embodiment terminal is arranged to 
communicate with an identification device, the identification device being provided 
with: 

25 • a photographic image of a person and 
• a microprocessor, 

the microprocessor comprising: 

> a processor, 

> a memory connected to the processor and comprising authentication data, and 
30 > interface means connected to the processor for communicating with a terminal, 

wherein the photographic image comprises steganographically hidden information, the 
content of which together with the authentication data allows authentication of the 
photographic image, 
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the processor being arranged to carry out at least part of the authentication of the pho- 
tographic image, 
the terminal being provided with: 

• a picture scanner to scan the photographic image and to generate image data, 

5 • a terminal interface allowing communication with the processor of the identifica- 
tion device, and 

• an image processor arranged 

> to receive the image data, 

> to analyze these image data in accordance with a predetermined image analysis 
10 procedure to derive the hidden information, and 

> to transmit at least the hidden information to the processor to allow the proces- 
sor to carry out the at least part of the authentication of the photographic image. 

Moreover, the invention relates to a method for authenticating a photographic 
image on an identification device, the identification device being provided with: 
15 • a photographic image of a person and 

• a microprocessor, 

the microprocessor comprising: 

> a processor, 

> a memory connected to the processor and comprising authentication data, and 
20 > interface means connected to the processor for communicating with an external 

device, 

wherein the photographic image comprises steganographically hidden information, the 
content of which together with the authentication data allows authentication of the 
photographic image, 
25 the method comprising the following steps: 

• scanning the photographic image and generating image data, 

• analyzing these image data in accordance with a predetermined image analysis pro- 
cedure to derive the hidden information, and 

• carrying out the authentication of the photographic image based on the hidden in- 
30 formation and the authentication data. 

Finally, the invention relates to data carriers provided with a computer program and 
to computer programs as such for such a method. 
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Hereinafter, the present invention will be illustrated with reference to some 
drawings which are intended to illustrate the invention and not to limit its scope. 

Brief description of the drawings 

Figure 1 is a schematic drawing of the system according to the invention showing 
a smart card and a terminal; 

figure 2 shows the functional units of the microprocessor of the smart card in a 
schematic way; 

figure 3 schematically shows how information can be hidden in a photographic 
image; 

figure 4 shows a flow diagram of the method according to the invention. 
Detailed description of the embodiments 

Figure 1 shows a smart card 1 provided with a photographic image 3 of the card 
holder. The smart card 1 is provided with an interface 5 shown to be made of metallic 
pads. However, the interface 5 may have any other form, e.g., an antenna hidden within 
the smart card 1 allowing for contactless communication with an external device. 

As shown in figure 2, the interface 5 is connected to a card processor 7 which is 
also connected to a card memory 9. 

Returning now to figure 1, the smart card 1 is, preferably, provided with one or 
more orientation signs 2 assisting a scanner during scanning the photographic image 3, 
as will be explained hereinafter. 

It is observed that figure 1 shows a smart card 1 but that the invention is equally 
applicable for other types of documents having a photographic image of the document 
holder and a processor arranged to communicate with an external device. 

The smart card 1 may be inserted into and removed from an opening 14 in a ter- 
minal 1 1 . The terminal 1 1 is provided with a picture scanner 13 and a connector 15. 

The picture scanner 13 is arranged such that it may scan the photographic image 
3 either during insertion of the smart card 1 into the opening 14 or after the smart card 
1 has been inserted entirely in opening 14. 
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The connector 15 will contact the interface 5 of the smart card 1 when the smart 
card 1 has been inserted entirely in the opening 14. Of course, when interface 5 is de- 
signed in another form, e.g. an antenna, the connector 15 is to be replaced by another 
type of interface arranged to communicate with interface 5. 

5 The picture scanner 13 is connected to a processor 17 which is also connected to 

the connector 15 and to a memory 19. 

Figure 1 also shows some input means, like a mouse 21, and a keyboard 23, al- 
lowing an operator to input data to the processor 17. A monitor 25 connected to the 
processor 17 is provided to allow the processor to display necessary information to the 

10 operator. Of course, any other kind of display means may be provided instead of or in 
addition to monitor 25. 

In an alternative embodiment of the terminal shown in figure 1, the processor 17 
is provided as a processing unit within the picture scanner 13. Then, the picture scanner 
13 is directly connected to connector 15 (or any other interface) by a direct link indi- 

1 5 cated with reference sign 1 6. 

Figure 3 schematically shows that the photographic image 3 is provided with 
additional information 4. The additional information 4 is added to the photographic 
image 3 such that it is invisible to the human eye. Moreover, the additional information 
4 may have such small dimensions that it is virtually impossible to be detected by auto- 

20 matic scanners if they do not know where to look for the additional information. The 
additional information 4 is added to the photographic image 3 by using steganographic 
techniques which are known to persons skilled in the art. 

It is observed that, in figure 3, the additional information 4 is shown on such an 
enlarged scale that it is visible but in practice it will not be visible to the human eye. 

25 Moreover, in a preferred embodiment, the individual dots of information 4 are distrib- 
uted over the entire image 3 to make it more difficult to find them. 

The additional information 4 may have no relation at all to the content of the 
photographic image 3. However, the photographic image 3 before being printed on the 
smart card 1 may be preprocessed in such a way that the additional information 4 is 

30 calculated in dependence on the content of the photographic image 3 such that the 
degree to which it is hidden in the photographic image 3 is as best as possible. 
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In accordance with the present invention, the card memory 9 is provided with 
authentication data. The content of this authentication data, together with the hidden 
information 4 allows authentication of the photographic image 3. 

In its simplest form, the authentication data has a one to one relation to the hidden 
5 information 4. However, the hidden information 4 may be present within the photo- 
graphic image 3 in cryptographically processed form, e.g., it may be provided with a 
cryptographic signature such that the validity of the hidden information 4 can only be 
checked by an apparatus knowing the key to the cryptographic signature. Such a key is, 
then, stored in the card memory 9. 
10 The hidden information 4 is such that it can be recognized by digitization of the 

photographic image 3 even if it is incomplete or otherwise impaired. The hidden infor- 
mation may have the form of a digital watermark. 

Checking the validity of the hidden information may be based on any kind of 
calculation using both the hidden information 4 and the authentication data in the card 
15 memory 9. 

As shown in figure 4, in order to allow for authentication, the card holder has to 
insert his or her smart card 1 into the opening 14 of the terminal 1 1 . During insertion or 
after completing the insertion, the picture scanner 13 scans the photographic image 3, 
while interfaces 5 and 15 may communicate with one another. The orientation signs 2 
20 may assist the picture scanner 13 in detecting where to search for the hidden 
information 4. The picture scanner 13 processes the photographic image 3 and 
generates image data which is sent to the processor 17, step 30. 

The processor 17 digitally processes the received image data, as well as the 
authentication data stored in the card memory 9 in accordance with a predetermined 
25 program. In accordance with the predetermined program, which is preferably stored in 
memory 19, the processor 17 separates the hidden information 4 from the photographic 
image 3, step 32, and uses the hidden information 4 to establish the authenticity of the 
photographic image 3, step 34. 

The authentication data received from the smart card 1 may be protected with any 
30 cryptographic means known to persons skilled in the art. Additionally, the data may be 
provided with a digital signature. 

The authentication process carried out by the processor 17 may depend on the 
authentication data received from the smart card 1 in such a way that for different 
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authentication data a different authentication process is carried out. This further en- 
hances security. 

,In an alternative embodiment, instead of the processor 17 in the terminal 1 1, the 
card processor 7 is arranged to carry out the authentication process. To that end, it 

5 receives the hidden information 4 by means of the terminal 1 1. 

However, since the card processor 7 and its memory 9 will only have a limited 
capacity, in practice, it will be preferred that both the processor 17 of the terminal 1 1 
and the card processor 7 carry out part of the authentication process. The card processor 
7 may, for instance, perform a final authentication step of the authentication process. 

10 In a further embodiment the card memory 9 may be provided with credential 

data, i.e., data indicating predetermined actions the card holder is allowed to do, e.g., 
entering a building or an area, debitting an account, etc. In that case, the card processor 
7 is, preferably, arranged to transmit these credential data to the processor 17 only 
when its own part of the authentication process has been carried out successfully. Thus, 

15 by receiving the credential data the processor 17 is informed that the authentication 
steps carried out by card processor 7 did not find any problems. When it does not 
receive the credential data the processor 17 knows that the authentication process has 
been unsuccessful. 

To further enhance the security, the authentication data stored in card memory 9 
20 may be related to one or more specific or general characteristics of the image 3 itself, 
like grey level, intensity distribution or image entropy. These parameters will be 
derived by the picture scanner 13 and transmitted to the processor 17. These parameters 
may be used by the processor 17 during the authentication process. However, in order 
to further enhance security, these parameters may be passed through the processor 1 7 to 
25 the card processor 7 which uses one or more of these parameters during carrying out its 
authentication steps. 

Instead of the picture scanner 1 3 establishing the value of one or more of these 
parameters, these parameters may be digitally stored in the photographic image 3. The 
digitized value of these parameters may have been printed after being encoded. Before 
30 these digitized values of these parameters are added to the photographic image 3 they 
may be encoded. 

In the embodiment described above, the terminal 11 is shown to include a 
memory 19. As is evident to persons skilled in the art memory 19 may comprise any 
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kind of memory type like RAM, ROM, EPROM, EEPROM, etc. or any combination 
thereof. For the purpose of the present invention the memory 19 need not necessarily 
be physically located within the terminal 11. 

Moreover, the processor 17 is shown to be one block. However, if preferred, the 
5 processor 17 may be implemented as several subprocessors communicating with one 
another each dedicated to perform a predetermined task. Preferably, the processor 17 is 
(or the subprocessors are) implemented as a computer with suitable software. However, 
if desired, they may be implemented as dedicated digital circuits. 

The method in accordance with the present invention is preferably implemented 
10 by suitable software. This software may be distributed by data carriers like CDROM's 
or through the Internet or any other data communication medium. 
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1 . Identification device (1) provided with: 
• a photographic image of a person (3) and 

5 • a microprocessor (8), 

the microprocessor (8) comprising: 

> a processor (7), 

> a memory (9) connected to the processor (7) and comprising authentication 
data, and 

10 > interface means (5) connected to said processor (7) for communicating with an 

external device; 

wherein said photographic image (3) comprises steganographically hidden information, 
the content of which together with said authentication data allows authentication of said 
photographic image (3). 

15 

2. Identification device (1) according to claim 1, wherein said identification device 
(1) is one of the following set of items: passport and plastic identification card. 

3. Identification device (1) according to claim 1 or 2, wherein said processor (7) is 
20 arranged to carry out at least part of said authentication. 

4. Identification device (1) according to claim 1, 2 or 3, wherein said hidden infor- 
mation is provided with a cryptographic signature. 

25 5. Identification device (1) according to any of the preceding claims, wherein said 
authentication data is at least partly related to a feature of the photographic image (3). 

6. Identification device (1) according to any of the preceding claims, wherein said 
hidden information is related to a feature of the photographic image (3). 

30 

7. Identification device (1) according to claim 5 or 6, wherein said feature is at least 
one of the following set of features: gray level, intensity distribution, and image 
entropy. 
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8. A terminal (1 1) arranged to communicate with an identification device (1), said 
identification device (1) being provided with: 

• a photographic image of a person (3) and 
5 • a microprocessor (8), 

the microprocessor (8) comprising: 

> a processor (7), 

> a memory (9) connected to the processor (7) and comprising authentication 
data, and 

10 > interface means (5) connected to said processor (7) for communicating with a 

terminal (11), 

wherein said photographic image (3) comprises steganographically hidden information, 
the content of which together with said authentication data allows authentication of said 
photographic image (3), 
1 5 said terminal being provided with: 

• a picture scanner (13) to scan the photographic image (3) and to generate image 
data, 

• a terminal interface (15) allowing communication with said processor (7) of the 
identification device (5), and 

20 • an image processor ( 1 7) arranged 

> to receive said image data, 

> to analyze these image data in accordance with a predetermined image analysis 
procedure to derive said hidden information, 

> to receive said authentication data from said memory (9), and 

25 > to carry out at least part of said authentication of said photographic image (3) 

based on said authentication data and said hidden information. 

9. A terminal according to claim 8, wherein said authentication comprises a set of 
predetermined steps stored in a terminal memory (19), the way in which said steps are 

30 carried out depending on said authentication data received from said memory (9) of 
said identification device (1). 
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10. A terminal (11) arranged to communicate with an identification device (1), said 
identification device (1) being provided with: 

• a photographic image of a person (3) and 

• a microprocessor (8), 

5 the microprocessor (8) comprising: 

> a processor (7), 

> a memory (9) connected to the processor (7) and comprising authentication 
data, and 

> interface means (5) connected to said processor (7) for communicating with a 
10 terminal (11), 

wherein said photographic image (3) comprises steganographically hidden information, 
the content of which together with said authentication data allows authentication of said 
photographic image (3), 

said processor (7) being aiTanged to carry out at least part of said authentication of said 
1 5 photographic image (3), 

said terminal being provided with: 

• a picture scanner (13) to scan the photographic image (3) and to generate image 
data, 

• a terminal interface (15) allowing communication with said processor (7) of the 
20 identification device (5), and 

• an image processor ( 1 7) arranged 

> to receive said image data, 

> to analyze these image data in accordance with a predetermined image analysis 
procedure to derive said hidden information, and 

25 > to transmit at least said hidden information to said processor (7) to allow said 

processor (7) to cany out said at least part of said authentication of said photo- 
graphic image (3). 



11. A terminal according to claim 8, 9 or 10, wherein said terminal has an opening 
30 (14) for receiving said identification device (1), both said picture scanner (13) and said 
terminal interface (15) being located within said opening (14). 
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12. A terminal according to any of the claims 8 through 1 1, wherein said image pro- 
cessor (17) is integrated within said picture scanner (13). 

13. A method for authenticating a photographic image (3) on an identification device 
5 ( 1 ), said identification device ( 1 ) being provided with: 

• a photographic image of a person (3) and 

• a microprocessor (8), 

the microprocessor (8) comprising: 

> a processor (7), 

10 > a memory (9) connected to the processor (7) and comprising authentication 

data, and 

> interface means (5) connected to said processor (7) for communicating with an 
external device, 

wherein said photographic image (3) comprises steganographically hidden information, 
1 5 the content of which together with said authentication data allows authentication of said 
photographic image (3), 
said method comprising the following steps: 

• scanning the photographic image (3) and generating image data, 

• analyzing these image data in accordance with a predetennined image analysis pro- 
20 cedure to derive said hidden information, and 

• carrying out said authentication of said photographic image (3) based on said hid- 
den information and said authentication data. 

14. A data carrier provided with a computer readable program for a method for 
25 authenticating a photographic image (3) on an identification device (1), said 

identification device (1) being provided with: 

• a photographic image of a person (3) and 

• a microprocessor (8), 

the microprocessor (8) comprising: 
30 > a processor (7), 

> a memory (9) connected to the processor (7) and comprising authentication 
data, and 
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> interface means (5) connected to said processor (7) for communicating with an 
external device, 

wherein said photographic image (3) comprises steganographically hidden information, 
the content of which together with said authentication data allows authentication of said 
5 photographic image (3), 

said method comprising the following steps: 

• scanning the photographic image (3) and generating image data, 

• analyzing these image data in accordance with a predetermined image analysis pro- 
cedure to derive said hidden information, and 

10 • carrying out said authentication of said photographic image (3) based on said hid- 
den information and said authentication data 

15. A computer program product for a method for authenticating a photographic 
image (3) on an identification device (1), said identification device (]) being provided 
15 with: 

• a photographic image of a person (3) and 

• a microprocessor (8), 

the microprocessor (8) comprising: 

> a processor (7), 

20 > a memory (9) connected to the processor (7) and comprising authentication 

data, and 

> interface means (5) connected to said processor (7) for communicating with an 
external device, 

wherein said photographic image (3) comprises steganographically hidden information, 
25 the content of which together with said authentication data allows authentication of said 
photographic image (3), 
said method comprising the following steps: 

• scanning the photographic image (3) and generating image data, 

• analyzing these image data in accordance with a predetermined image analysis pro- 
30 cedure to derive said hidden information, and 

• carrying out said authentication of said photographic image (3) based on said hid- 
den information and said authentication data. 
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Fig 4 



scanning the photographic image 3 and generating 
image data 



30 



4 



32 



analyzing these image data in accordance with a 
predetermined image analysis procedure to derive the 
hidden information 



4 



34 



carrying out the authentication of the photographic image 
3 based on the hidden information and the authentication 

data. 
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